Web Document Classification based on Tagged-Region Progressive Analysis

In this paper, we propose an intelligent web document classification method, called TAgged-Region Progressive Analysis (TARPA). Instead of parsing the whole content of the web page while classifying a web document, TARPA parses the document into finer structured Tagged-Regions and extracts fewer and the most important regions to analyze and classify. If the few important tagged regions are not sufficient to allow TARPA to classify the document, other important regions and linked pages can be used for analysis progressively to enhance the classification performance. TARPA possesses two stages: learning stage and classification stage. The learning stage discriminates the importance of tags or pairs of tags, and the classification stage follows the importance order of tags to analyze the document. As a result, TARPA can classify a web document using few contents while with higher classification rate and shorter processing time. Experiments show that 94% of the testing web documents can be correctly classified by only feeding the TARPA classifier with 30% to 50% of the document contents. Keyword(s): TARPA, web document classification, tagged-region, progressive analysis, information retrieval, Nature Language Processing (NLP). Web Document Classification based on Tagged-Region Progressive Analysis Abstract In this paper, we propose an intelligent web document classification method, called TAgged-Region Progressive Analysis (TARPA). Instead of parsing the whole content of the web page while classifying a web document, TARPA parses the document into finer structured Tagged-Regions and extracts fewer and the most important regions to analyze and classify. If the few important tagged regions are not sufficient to allow TARPA to classify the document, other important regions and linked pages can be used for analysis progressively to enhance the classification performance. TARPA possesses two stages: learning stage and classification stage. The learning stage discriminates the importance of tags or pairs of tags, and the classification stage follows the importance order of tags to analyze the document. As a result, TARPA can classify a web document using few contents while with higher classification rate and shorter processing time. Experiments show that 94% of the testing web documents can be correctly classified by only feeding the TARPA classifier with 30% to 50% of the document contents.